[Feature][Flink] Add flink version 1.20.1 support #9576

yzeng1618 · 2025-07-15T11:53:13Z

Purpose of this pull request

This pull request fixes the Flink 1.20 compatibility issue in the SeaTunnel Flink translation layer. The current implementation uses reflection to access internal StreamingRuntimeContext from SourceReaderContext, which is fragile and may break with Flink version updates.

Does this PR introduce any user-facing change?

No. This is an internal implementation fix that maintains the same public API behavior. Users will continue to use the same SeaTunnel connector APIs without any changes. The metrics functionality remains identical from the user perspective.

How was this patch tested?

Manual Testing:
Built SeaTunnel with Flink 1.20 dependencies
Ran sample connector jobs to verify metrics collection
Confirmed no reflection-related errors in logs
Validated metrics appear correctly in Flink Web UI

Check list

If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update plugin-mapping.properties and add new connector information in it
2. Update the pom file of seatunnel-dist
3. Add ci label in label-scope-conf
4. Add e2e testcase in seatunnel-e2e
5. Update connector plugin_config

Hisoka-X · 2025-07-15T11:54:04Z

cc @TyrantLucifer

TyrantLucifer · 2025-07-15T11:55:31Z

Good pr

...-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/FlinkExecution.java

...ation-flink-20/src/main/java/org/apache/seatunnel/translation/flink/source/SplitWrapper.java

Hisoka-X

We need new flink 1.20.1 test container to verify it. https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/flink

…n-flink-common

...-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/FlinkExecution.java

...-20/src/main/java/org/apache/seatunnel/translation/flink/metric/FlinkAccumulatorCounter.java

...flink-20/src/main/java/org/apache/seatunnel/translation/flink/metric/FlinkMetricContext.java

Hisoka-X · 2025-10-01T12:39:50Z

waiting test case passes.

Carl-Zhou-CN · 2025-10-13T10:28:37Z

@yzeng1618 CI failed. Please check

…PL-support1.20.1-test

yzeng1618 · 2025-10-20T10:07:05Z

waiting test case passes.

@yzeng1618 CI failed. Please check

Currently, it seems that there is an issue with the CI (Continuous Integration) process of the dev (development) environment itself.

Carl-Zhou-CN · 2025-10-21T07:15:13Z

@Hisoka-X @zhangshenghang It seems there's a problem with the mirror image. Do you have any updated information?

zhangshenghang · 2025-10-23T13:07:01Z

@Hisoka-X @zhangshenghang It seems there's a problem with the mirror image. Do you have any updated information?

It should be working normally now, and other users haven't reported this issue.

zhangshenghang · 2025-10-23T13:07:44Z

Please merge dev. it fix the timeout issue.

# Conflicts: # .github/workflows/backend.yml # seatunnel-core/seatunnel-flink-starter/seatunnel-flink-starter-common/src/main/java/org/apache/seatunnel/core/starter/flink/execution/SinkExecuteProcessor.java

yzeng1618 · 2025-10-24T02:10:14Z

Please merge dev. it fix the timeout issue.

Have merged

dybyte · 2025-10-26T13:30:59Z

Please pull the latest changes and try again.

yzeng1618 · 2025-11-05T01:26:24Z

@Hisoka-X @TyrantLucifer @Carl-Zhou-CN @zhangshenghang After adapting the new code and fixing the CI (Continuous Integration) process, the modifications have now been completed. Could you please help review this

zengyi added 6 commits July 3, 2025 17:40

[improve][start] version1.0

cd26913

[improve][start] version2.0

9dba128

[improve][start] version3.0

e020b82

[improve][start] version4.0

f870cef

[improve][start] version5.0

bef73ee

[improve][start] support 1.20.1 demo

777d513

github-actions bot added core SeaTunnel core module flink api labels Jul 15, 2025

Hisoka-X changed the title ~~[improve][start] Seatunnel version support 1.20.1~~ [Feature][Flink] Add flink version 1.20.1 support Jul 15, 2025

TyrantLucifer reviewed Jul 15, 2025

View reviewed changes

...-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/FlinkExecution.java Outdated Show resolved Hide resolved

zengyi added 2 commits July 16, 2025 11:56

[improve][start] refactor the code

bc46786

Merge branch 'dev' into BDPL-support1.20.1-test

87b0319

nielifeng requested a review from Copilot July 17, 2025 09:46

This comment was marked as outdated.

Sign in to view

zengyi added 5 commits July 19, 2025 17:38

[improve][start] Remove unnecessary classes and comment content.

4229fc1

[improve][start] Remove unnecessary classes and comment content.

943c501

[improve][start] Remove unnecessary classes.

0b6ad1f

[improve][start] Fix some details

e3afc1d

[improve][start] Fix some details in the flink translation module

4bc665d

yzeng1618 requested a review from TyrantLucifer July 19, 2025 10:57

Carl-Zhou-CN reviewed Jul 26, 2025

View reviewed changes

...ation-flink-20/src/main/java/org/apache/seatunnel/translation/flink/source/SplitWrapper.java Outdated Show resolved Hide resolved

yzeng1618 requested a review from Carl-Zhou-CN July 27, 2025 09:50

Hisoka-X reviewed Jul 31, 2025

View reviewed changes

zengyi added 3 commits August 2, 2025 17:54

[improve][start] removed content consistent with seatunnel-translatio…

2a6c2ef

…n-flink-common

[improve][start] Optimized the details

96d9ebc

[improve][start] add Flink20Container

baf00f6

github-actions bot added the e2e label Aug 3, 2025

[Feature][hive] increase timeout

4ff372f

Hisoka-X reviewed Sep 28, 2025

View reviewed changes

zengyi added 2 commits September 29, 2025 14:51

[Feature] fix cdc bug

994dfd6

[Feature][flink1.20] remove code

d051387

yzeng1618 requested a review from Hisoka-X September 29, 2025 10:14

[Feature] fix accumulatorCounter

29adacd

yzeng1618 and others added 4 commits October 14, 2025 17:27

Merge branch 'dev' into BDPL-support1.20.1-test

e8a5ddd

[Feature] restore

af0e538

Merge remote-tracking branch 'origin/BDPL-support1.20.1-test' into BD…

54f023c

…PL-support1.20.1-test

[Feature] fix ci

196ff9f

[Feature] remove throw

ca90791

Merge branch 'dev' into BDPL-support1.20.1-test

38a00b9

# Conflicts: # .github/workflows/backend.yml # seatunnel-core/seatunnel-flink-starter/seatunnel-flink-starter-common/src/main/java/org/apache/seatunnel/core/starter/flink/execution/SinkExecuteProcessor.java

[Feature] add timeout-minutes

b6a7a6b

yzeng1618 and others added 8 commits October 27, 2025 15:28

Merge branch 'apache:dev' into BDPL-support1.20.1-test

1a9e14e

[Feature] increase timeout

8a4edea

[Feature] fix SourceExecuteProcessor

22d9bea

Merge branch 'apache:dev' into BDPL-support1.20.1-test

3a3fddf

[Feature] increase timeout

c25ba7e

[Feature] flink 1.20.1 translation support schema evolution(apache#9867)

18f36e0

[Feature] increase all-connectors-it-6 timeout

fb63be2

[Feature] modify openjdk:8 to openjdk:8u342

ad8a6fb

[Feature][Flink] Add flink version 1.20.1 support #9576

Are you sure you want to change the base?

[Feature][Flink] Add flink version 1.20.1 support #9576

Uh oh!

Conversation

yzeng1618 commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

Uh oh!

Hisoka-X commented Jul 15, 2025

Uh oh!

TyrantLucifer commented Jul 15, 2025

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Hisoka-X left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Hisoka-X commented Oct 1, 2025

Uh oh!

Carl-Zhou-CN commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yzeng1618 commented Oct 20, 2025

Uh oh!

Carl-Zhou-CN commented Oct 21, 2025

Uh oh!

zhangshenghang commented Oct 23, 2025

Uh oh!

zhangshenghang commented Oct 23, 2025

Uh oh!

yzeng1618 commented Oct 24, 2025

Uh oh!

dybyte commented Oct 26, 2025

Uh oh!

yzeng1618 commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

yzeng1618 commented Jul 15, 2025 •

edited

Loading

Carl-Zhou-CN commented Oct 13, 2025 •

edited

Loading